As mathematicians, we experiment constantly. Just try some shit for heaven's sake!
I'm talking about several different kinds of experimentation here. I'll look at three kinds below. (This is not exhaustive of course, and there are probably other viewpoints to take).
Just work out a few problems. This is what we'd like our students to do most often when we say "experiment." Try similar types of problems and see what happens. Make links between the ideas. Try solving x^2+3x+4=0, x^2+3x+3=0, x^2+3=0, and so on, and see what different kinds of solutions you get -- before asking for the theorem ("rule") about what kinds of solutions you get.
Why don't students do this? If math is a series of hazily-understood rules, it's safer to just follow those received rules.
Mess with your assumptions. This is a higher level of sophistication but can still apply even to problems like the quadratic equations above. At some point in school we learn that you can't take the square root of a negative number. This is just a lie adults tell us to protect us from something they believe to be scary. All around the world, though, there's some kid every day who says, "Why not?" That's the right question to ask. Why not?
Sometimes there are good reasons "why not." In mathematics eventually one develops a sense of how things "should be" and it is disturbing when violations are found. This is where interesting things happen. But messing with these assumptions is also very useful. Why can't we take square roots of negative numbers? Well, ... um... in the end, no reason -- so we discover imaginary numbers. Why can't we let time go to infinity in this dynamical system and allow negative populations of gazelles, if only in our minds? Well... no reason -- and then we discover something about stable solutions and that our model actually works for an engineering application. Why can't we take the quotient of a geometric space this way instead of that way? Well.... now we develop Chow quotients, GIT quotients, symplectic quotients, stacks.
Gather data like a scientist. Experimentation by hand or by computer can be deeply valuable. Programming the calculations -- often the only way to gather a lot of data in math -- also forces a different point of view that can be illuminating. (Comparing Sage and Macaulay2's treatment and implementation of Schur functions, for instance, is interesting.) The data you get at the end can be REALLY interesting! You can disprove conjectures quickly by finding counterexamples. You can get a suggestion for a new theorem by noticing a pattern (why are these numbers all even? all 0 mod 4? all prime?). You can discover unexpected connections to other areas of mathematics (the results of the combinatorial experiments gave me formulas that solve this differential equation....?!). You can publish things like "this conjecture has been checked for all n less than 17" or "all n less than 16,092,123".
Pure math involves proof: this is what differentiates it from the other sciences. We should not forget, though, that some of the initial investigatory impulses we have share a lot with the sciences. We shouldn't let our students forget, either.