This is G o o g l e's cache of http://www.ubercart.org/forum/support/463/not_your_cart_ubercart_reporti... as retrieved on Aug 4, 2007 00:38:18 GMT.
G o o g l e's cache is the snapshot that we took of the page as we crawled the web.
The page may have changed since that time. Click here for the current page without highlighting.
This cached page may reference images which are no longer available. Click here for the cached text only.
To link to or bookmark this page, use the following url: http://www.google.com/search?q=cache:SJpciVGEQkUJ:www.ubercart.org/forum...

Google is neither affiliated with the authors of this page nor responsible for its content.
These search terms have been highlighted: druru
Ubercart

* Home
* Documentation
* Forums
* Issues
* Livetest

Home » Forums » Ubercart » Support
not your cart -> ubercart reporting
Submitted by psi-borg on Wed, 03/21/2007 - 11:11
psi-borg

Posts: 117
Joined: 03/20/2007
Getting busy with the Ubercode.R2D2, I am your father.

very googlish thing you're doing with [no con]census reporting:

"Übercart reports to www.ubercart.org every so often to scan for module updates and news alerts. While this option is set to Yes, some anonymous e-commerce data is also reported for fun statistical purposes. No personal identifiable information will ever be published."

what exactly would be reported? what is reported regardless of which census option chosen?

sure data mining can be great - what exactly do you collect and plan to do with it?

if you plan to open source the data mining functionality (make it a distributed system through all ubercarts [which would be impressive]) i'd say why not...
--

zeni.mobi

* Add new comment

permalinkWed, 03/21/2007 - 11:26
Ryan

Posts: 1693
Joined: 09/26/2006
AdministratorCode Monkey Head - I eat bugs.

lol @ [no con]census reporting. You should be able to look in the cron_hook of uc_store.module to see what's being looked for and reported. I believe it's module versions and maybe PHP version if the census settings are turned off. Honestly, we're doing nothing with the data right now, but we hope to be able to send out security information that would show on the store administration pages. If that's not popular, we can revise those plans. As sites start going live, we hope to get a tracker like osCommerce has showing how many sites are running Übercart.

If "yes" is chosen for the census page, it's reporting a total on orders completed for the last x days that is added into the total figure so we can get another fun little stat tracker... how much money have people made with Übercart. Sticking out tongue That can be turned off, though.

Thoughts?

* reply

permalinkWed, 03/21/2007 - 11:35
jonathanchris@d...

Posts: 68
Joined: 03/09/2007
Bug Finder

Personally I like it! But I understand those that might feel uneasy about it. I know when I first saw it, I was taken back a bit (the paranoia set in). Who are these people? And what do they want with my information? But soon I realized how helpful it would be to get alerts and updates if/when my setup needed it. Sometimes a module will get updated on drupal and I won't realize it for a while, this is a convenient way to be updated to that fact. If someone is running a site, they should at least be going to their own site on a daily basis, but might not visit this site or d.o on a daily basis.

As far as the # of sites running ubercart, and the # of sales, etc. I like the idea of checking out those stats, it would be kind of cool to see. But I definitely think it should be an option left up to the site administrators. Some people won't want this feature.

* reply

permalinkWed, 03/21/2007 - 11:50
Lyle

Posts: 569
Joined: 09/26/2006
AdministratoreLiTe!

It's not on the front page anymore, but there is now an Update Status module for Drupal. We were going to do something very similar for Übercart, but it seems we were beaten to the punch.

We were hoping that people would be interested in statistical information about all the Übercart sites. But, if they're not, they can certainly opt out of sending business information.

Do not fear, we use our powers for good.

* reply

permalinkWed, 03/21/2007 - 11:54
Ryan

Posts: 1693
Joined: 09/26/2006
AdministratorCode Monkey Head - I eat bugs.

"Do not fear, we use our powers for good."

Yikes, now you're really making us sound Google-ish. Sticking out tongue

* reply

permalinkWed, 03/21/2007 - 13:38
psi-borg

Posts: 117
Joined: 03/20/2007
Getting busy with the Ubercode.R2D2, I am your father.

ok so i missed the broad side of the...

i'm not against sharing data, the key word being share. i'm actually for anonymous data collection to try to make things more predictable... and so the reference to distributed data mining, where each store owner that opts in to sharing their data can also receive the benefit of global data mining functions, much the way google operates with the google toolbar for example. you share your search terms, google gets better for you locally and (to a lesser extent) globally for all users (i'm not sure about their particular distributed computing algos, or if its distributed at all).

lets say, for example, we define each drupal/uc site as a dynamic vector in the vector space of all drupal/uc sites. each vector has component values, like

v(t) = {module_x_installed = 1, uc_census = 1, bg_color = fff, monthly_sales = 52968.35, ...} // had to use set notation due to format interpretation

if monthly_sales component is the target value, u.o can have an ongoing time series optimization program running that would give values for fitting parameter (other components) combinations that has historically yielded values within a given range.

you might have several math models competing in parallel for highest r^2 values if prediction methods have no clear best candidate (like running a ga on all models, not necessarily with the same parameter space).

the goal is that if we share data as a community, that would make us as a community much stronger... i'm sure you can see how this kind of functionality would be very attractive to people on the fence of which cart to use.
--

zeni.mobi

* reply

permalinkWed, 03/21/2007 - 16:46
druru
Posts: 225
Joined: 03/21/2007
Brain Stormer

Well, I guess anyone can go in and modify that hook to do whatever they want so that's good. Even modify it for their own data collection purposes.

One thing that bothers me a little is that when you disable the setting it STILL sends information. That's a little misleading for the paranoid out there. It's highly possible that some uc implementations don't want you knowing about their sales stats.

Also and maybe you guys don't care now but might in the future depending on the power of your servers, you're going to start getting hit with lots of http requests to your servers as uc takes off and gets implemented in the real world. just something to think about.

Maybe 3 options would be better:

1) Enabled - send all data including sales figures
2) Partially Enabled - only send php and module info for update status
3) Disabled - No information sent (this would disable the hook completely)

As lyle said, there is now an update status module available which checks for all drupal modules and i believe that's all it does.

* reply

permalinkWed, 03/21/2007 - 16:51
Ryan

Posts: 1693
Joined: 09/26/2006
AdministratorCode Monkey Head - I eat bugs.

Yeah, it may be worth it to take module stuff out of there, which is all it's doing when the option is set to "No" anyways. Since we're using Drupal's CVS repository, it may be worth it to outsource that in favor of the release module. I hadn't thought about the server load issue... I'm sure Andy would be interested in considering that.

* reply

permalinkWed, 03/21/2007 - 17:11
psi-borg

Posts: 117
Joined: 03/20/2007
Getting busy with the Ubercode.R2D2, I am your father.

yah, turned it off and youre still trying to harvest...

$response = drupal_http_request('http://www.ubercart.org/statistics/gather', array('Content-Type' => 'application/x-www-form-urlencoded'), 'POST', http_build_query($vars, '', '&'));

http_build_query() expects at most 2 parameters, 3 ...

ditto on the "if youre thinking of building trust, this doesnt help..." bandwagon. also setting this to off by default would help.
--

zeni.mobi

* reply

permalinkWed, 03/21/2007 - 17:17
Ryan

Posts: 1693
Joined: 09/26/2006
AdministratorCode Monkey Head - I eat bugs.

We'll look into it tomorrow. I thought it worked as I advertised. Sticking out tongue

* reply

permalinkWed, 03/21/2007 - 19:05
psi-borg

Posts: 117
Joined: 03/20/2007
Getting busy with the Ubercode.R2D2, I am your father.

im in the 'all fields should explicitly be agreed to' camp - check_all can be used for large sub_sets.
--

zeni.mobi

* reply

permalinkThu, 03/22/2007 - 12:57
Andy

Posts: 258
Joined: 09/27/2006
Administrator

I understand the people may be concerned about the census reporting, so I will try and explain what it does and why.

What the census currently reports:
The census reporting by default sends "basic" information including it's unique id (issued by Drupal on the first census report), ip address, host name, PHP version, Drupal version, Ubercart version, and all drupal module names and versions installed. There is no option to turn this level of reporting off. There is an option to send the following additional information; total products, total orders, total sales dollars. This additional information is enabled by default, but can be disabled by the administrator.

What the census should report:
The census "basic" information should include it's unique id (issued by Drupal on the first census report), ip address, administrator email address, host name, PHP version, Drupal version, Ubercart version, and only Ubercart module names and versions installed.

Why the census currently reports:
With a unique id, Ubercart.org can keep the rest of the information accurate and handle issues like multiple sites running on the same ip. The ip address and host name allow Ubercart.org to send notifications to administrators about out of date modules and or security risks. PHP version, Drupal version, and Ubercart module versions help us to recognize when a site is running old modules or modules with security risks. The additional information about products orders and sales are useful for judging the actual success of Ubercart.

What Ubercart will do with the information
Ubercart.org will use the basic information to notify administrators of updates and security risks to their sites via email, and to publish anonymous statistics about the number of Ubercart sites on Ubercart.org. We feel that a security risk to an e-commerce site is a greater risk to the administrator then a security risk to a blog. We feel that passive notification by posting the risk on Ubercart.org is not enough. We want to be able to email administrators if and when this happens. Ubercart.org will use the optional information to present "fun" completely anonymous statistics on Ubercart.org something like "Ubercart is currently in use on 1,068,349 sites and has generated 953,380,203 sales for a total of $51,676,568,546 in sales." (Why not think big Smiling ) Ubercart.org will never disclose any specific information regarding any particular site.

Why do the Uber-developers created the census report, and why some information is not optional
We think it is important to be able to proactively warn site administrators about security issues.
We have worked long and hard for six months. I have paid for two full time developers (and recently hired a third) on this project, not to mention the nights and evenings we have each individually contributed. We have made the fruits of all this effort available to you and the world for free. The only condition I place on using this software is that you let me know you are using it via the census report. The only request I have is that you send the optional information so we can have the satisfaction of knowing that our hard work is paying off for you and is a success.

My primary goal in sponsoring and developing Ubercart is to create a superior cart/CMS for the e-commerce sites I maintain. My secondary goal is to provide it to the open source community as a way of saying thanks to all the people who have developed open source software I have used. I encourage open discussion of the census report, I intend to prominently display what it does and doesn't do, and I intend to consider every argument for change in the reporting. That being said, I currently feel the census report is a fair and reasonable aspect of Ubercart and I doubt anyone will be able to convince me that it is unfair for ask this one thing in exchange for all we have done.
Peace,
Andy

* reply

permalinkThu, 03/22/2007 - 13:45
jonathanchris@d...

Posts: 68
Joined: 03/09/2007
Bug Finder

Preach on Brother! Sounds good to me! Smiling

Oh and thx!

* reply

permalinkThu, 03/22/2007 - 15:35
psi-borg

Posts: 117
Joined: 03/20/2007
Getting busy with the Ubercode.R2D2, I am your father.

andy, my friend, i don't think owners would be opposed to all census reporting.

but perhaps you should ask yourself: am i more or less likely to use something that gives me less choice or more difficult to opt out? make global reporting decisions easy to implement with check_all and company, but give us choices with the details. defaults should favor privacy.

i say, give the census special attention; don't bury it at the leaf of some configuration tree. as i've alluded to above, uc_.census has the potential to be a great asset to the uc community if it's used for helpful datamining (and decision support) operations for owners collectively.

if you are reasonable with us, uc systems will have a much better chance of approaching $51,676,568,546 in sales.
--

zeni.mobi

* reply

permalinkThu, 03/22/2007 - 16:53
druru
Posts: 225
Joined: 03/21/2007
Brain Stormer

few thoughts:

1) to my knowledge, no other drupal module does what census does (that doesn't mean that none do but to my knowledge none do). i think you are breaking tradition with the original project. i'm not saying it's good or bad just that it's new territory and the community may be apprehensive. Or you might find all kinds of modules starting to want to do this.

a) Would you use drupal if every module you installed reported back to it's project owner?
b) Would it degrade your local install performance?
c) If you answer no to q1 or yes to q2, then why should you be given exclusive right to be able to do what you don't want others in the community doing?

It brings up the whole issue of preference.

2) whether it was intended or not, i think your *initial* failure to clearly disclose what the census did initially might be construed by some to wonder what your intent actually is with collecting the information.

3) For a moment, regardless of your investment in the project, ask yourself this:

Does recieving junk mail piss you off?
Do you think every time you sign up for something you should automatically be "opted out" and then have to manually "opt in" to recieve that crap?

If you answer yes to either of those questions, and my thought is your answer is yes, you can see why some people might want to be automatically opted out.

It brings up the whole thing of preference. I don't want others to do it to me but yet it's ok for me to do it to others.

Now your investment in the project may force you to change your mind and that's ok. But you've got to get into the mindset of the folks that want to use this stuff too. That's going to be a marketing decision you'll have to make.

4) the way you've implemented this appears to be "push" oriented vs "pull". That is the more invasive (less private) of the two approaches.

For instance, if you are concerned about the updates & security of installations out in the field (e.g. you are doing this for our benefit), do you need to collect any LOCAL info to make that work?

I believe the way uc works now is PUSH. The local installed version sends you LOCAL information then you process it and send it back a reply. That gives you the option to collect more data about the local site.

However, it could easily have been implemented as PULL:

a) local site asks your site what teh current versions are or if there are security updates
b) you send YOUR info back to the local site
c) the LOCAL site compares it with it's own local information and prints "you need to update or security fix needed", etc.

Same end result but withOUT sending you any LOCAL information.

Just pointing that out. There's always different ways to do stuff.

5) Somewhere inside i'm an opt out guy by default and only prefer to opt in if i choose to do so.

Here's why. That sales information may be presented as innocuous but i'd really prefer for someone to not know how well or not well my business is doing? It's nobody's business.

Are you willing to share how well your business is doing with me? If the answer is no then why should i share my information with you?

fwiw, it's very easy for you to use the ip adddress of any installed uc system out there to find the business name and tie the business to it's sales. That's uncool. Even if you say won't do it. Maybe in the future you will change your mind.

Why don't you simply collect the number of installs. That's a good enough figure to base the health of UC. That can be achieved with the pull method i stated earlier.

6) Ultimately it's your module, you're going to do whatever you want in the end so the final decision remains with you. It sounds like you already have your mind made up from your previous post.

So consumers of your module will have to decide whether they want to live with that or not.

And more often than not, they will because they can't write this stuff themselves Eye-wink.

Then again they might not...

NOTE:

there's no animosity in my points above. just thought i'd share some of those with you to help you see how some might percieve what you guys are doing and open it up for debate. As i said earlier, it's your effort. And if you feel doing this justifies your investment have at it Smiling.

* reply

permalinkThu, 03/22/2007 - 17:22
druru
Posts: 225
Joined: 03/21/2007
Brain Stormer

Why the census currently reports:

The additional information about products orders and sales are useful for judging the actual success of Ubercart.

Does that additional information help display the actual success of UC, the business owners themselves or both?

Purely Hypothetical Examples:

A) there might be 1 million ubercart installations. 999,999 of them sell 0 products but 1 of them sells a billion dollars worth of goods.

B) Or there's only 1 UC installation and it sells 1 billion dollars worth of goods.

Your results are going to report 1 billion in sales. Does that accurately reflect what's going on in the real world at how successful UC is? Not in those examples.

What if in case B the guy sells the cure to cancer? he could've used OsCommerce and he's going to be raking in the dough regardless.

If a product is good it sells right? In this case UC is downloaded and used a lot. imho, the number of installs really determines the success of UC. If it's as rock solid as you guys think it's going to be, it will kick booty over osc and joomla and all the rest of em combined.

imho, what people do with UC is somewhat their own doing? Are you responsible for the success another person has who comes up with the creative idea for a product he sells using UC? Or the guy that actively markets his products a certain way so that people come to his UC in the first place? An UC just sits there until it's populated with products and creatively marketed. Kind of like "if a tree falls in teh forest do you hear it"? I'm not sure but i don't think so.

When Bill Gates kicks the bucket, he will be associated as the man that created microsoft the x billion dollar company. I'm not sure but his cart or accounting sw probably won't come up in his eulogy (even though they were undoubtedly vital parts to his success) Eye-wink

Either way, some more points to think about.

* reply

permalinkThu, 03/22/2007 - 20:59
Ryan

Posts: 1693
Joined: 09/26/2006
AdministratorCode Monkey Head - I eat bugs.

Hey druru... don't have a whole lot of feedback on your post other than to say you do make some valid points. I'll just say that the reason the feature wasn't fully documented was because 95% of the features aren't fully documented and probably half aren't documented at all. Eye-wink

* reply

permalinkFri, 03/23/2007 - 08:12
Brian

Posts: 31
Joined: 10/17/2006
Administrator

I think it is essential to make sure that the security and module updates are provided directly to the store owners. I have seen a few open source projects get a bad security reputation because of very serious bugs that are fixed in the current release but still exist in the thousands of old installations. I personally won't use phpbb because of some serious security problems it caused on one of my sites. Of course the developers had fixed it long ago so it was my fault, nevertheless, I still won't use it because I don't have time to keep up with every change.

Most store owners (obviously not the ones checking out this alpha release) just install a store and keep running on it for years without upgrading. When you are dealing with money and credit cards, there needs to be some way to proactively alert them that there is a security issue that needs to be addressed. This is even truer now that Visa and Mastercard are making the merchants liable for security breaches on their sites. Whether that method is push or pull, I don't know, but it needs to exist and it should be very difficult for the average user to turn off.

I personally would like to see the ability to automatically install/update modules with one click like Trixbox. (I remember joomla having some very easy way of installing modules too??)

As far as the statistics data, I actually check the statistics page most days just to see what is going on. That is part of the fun, but since it is just fun, I do think that store owners should be able to turn it off. Just include a note that explains what it does and ask them to leave it on so that you can keep accurate statistics on ubercart.

I think statistics information is primarily valuable from a marketing standpoint to give prospective store owners confidence in switching to ubercart. If you see that there are 50,000 stores and they have sold one billion dollars (or whatever Andy said), that gives them confidence that this software works and isn't going to disappear any time soon. Heck, you might even get a major site to switch if they can see those types of results.

It would be nice to add something that will (optionally) automatically post just your site, site name, and slogan to the live sites section of ubercart.org instead of having to add it manually. This would help sites using ubercart to get a free link from ubercart.org and show up better in search engines.

Just my thoughts but you're skinning this cat. I'm just holding the legs.

* reply

permalinkFri, 03/23/2007 - 08:31
Brian

Posts: 31
Joined: 10/17/2006
Administrator

Just a thought on the statistics:

You could make it post with just a unique id (not the same one as the security data id) so that there isn't any connection between the sales statistics and the actual site. That would make it anonymous so that it wouldn't be connected with site, but would still allow the statistics to be added accurately.

* reply

permalinkMon, 03/26/2007 - 10:04
Andy

Posts: 258
Joined: 09/27/2006
Administrator

After reading everyone's opinions on this topic I have changed my mind on a few aspects of the census reporting.

Right now administrators can opt out of the advanced reporting data, but not the basic data. I think we should give administrators the option to disable all census reporting.
Currently the census data is enabled by default and the administrator has to opt out. In the future I want Ubercart installation to force the administrator to make a choice as to whether census reporting is enabled and if so, what level of reporting is enabled. Barring this feature, I continue to believe the census reporting should be opt out. If the administrator does not care enough about this to go opt out, then it doesn't matter.

Also we will change the census data so that we do not collect the IP address. We did this initially in an attempt to separate different installations of Ubercart. We have since created the unique ID which solves this problem. Also. the Host name should be optional. We were collecting this in oder to compile a list of Ubercart installations on Ubercart.org. This will be a list of links to sites running Ubercart which would help with search engine listing for those sites which we consider to be a benefit for the administrator. Removing the IP address and host name will make the census data completely anonymous which should alleviate some of the big brother concerns.

Peace,
Andy

* reply

permalinkTue, 03/27/2007 - 03:42
psi-borg

Posts: 117
Joined: 03/20/2007
Getting busy with the Ubercode.R2D2, I am your father.

to some of dru's points:

1) >> find . -name "*" | xargs grep -i "drupal_http_request"

returns only uc file matches within /sites/all/modules/

note: i am rather new to drupal and have only 29 mods living in that directory

2) uc uses POST (more info, less secure v. GET ) for drupal_http_request::$method

3) yeah, man you don't want my site stats included if the uc site population is small - because i'll definitely skew your results for the worse. ^^

(googling "E-commerce powered by Übercart, the" returns 12 results)

====================

andy, i look up to you as a big brother, regardless ^^
--

zeni.mobi

* reply

permalinkThu, 07/19/2007 - 05:50
insipiens

Posts: 19
Joined: 07/19/2007

the following were activated by default:
" Number of fulfilled orders
Number of products sold
Total sales "

i.e. if a user doesn't manually walk through that configuration section and change it, the sales data is sent.
Guys, that is unacceptable, and i'm quite sure it's even illegal in many countries. The fact that those fields are marked as "Opt-in data" turns out to be completely false since they are all activated by default. Opt-in means that the user must explicitely and knowingly opt in. As is, that's not the case.
You really should change that.

* reply

permalinkThu, 07/19/2007 - 08:07
Ryan

Posts: 1693
Joined: 09/26/2006
AdministratorCode Monkey Head - I eat bugs.

Well, if your definition of opt-in only includes checking a box, then I suppose these aren't opt-in. But if, like a lot of software, consent can be granted simply by installing software then they are opt-in. (And this is a documented feature with instructions for turning it off... should you ever "change your mind." Eye-wink ) But that's splitting hairs. The name of the fieldset has been changed to Optional data, which we really like better anyways.

Thanks for bringing it to our attention. Smiling

* reply