Patrick, I have the same problem here.

I am trying to setup a rule as follows.

"Check an order's number of products"
Pick the the products that I want
Set the count value and comparison

The rule won't run when I click on get shipping quote
however if I go back and change products to "All Products"
The rule will run fine..